Growing Neural Network Trees Efficiently and Effectively

Neural network tree (NNTree) is a hybrid learning model with the overall structure being a decision tree (DT), and each non-terminal node containing an expert neural network (ENN). Generally speaking, NNTrees outperform conventional DTs because better features can be extracted by the ENNs, and the performance can be improved further through incremental learning. In addition, as we have shown recently, NNTrees can always be interpreted in polynomial time if we restrict the number of inputs for each ENN. Currently, we proposed an algorithm which can grow the tree automatically, and can provide very good results. However, the algorithm is not efficient because GA is used both in re-training the ENNs and in creating new nodes. In this paper, we propose a way to replace GA with the back propagation (BP) algorithm in the growing algorithm. Experiments with several public databases show that the improved algorithm can grow better NNTrees, with much less computational costs.